Human Action Recognition and Shape Segmentation-Recognition

Human Action Recognition. Human action recognition has broad range of applications such as video search, sports analysis, human robotics interactions, and health care. Our work is organized in two directions: 1) detailed pixel-level ‘motion and pose’, focusing on close interactions among people; 2) action recognition focusing on goal oriented motion, simplified as ‘action = motion + intention’. In “Detecting Unusual Activity in Video” (cvpr2004), we demonstrated that using large amount of un-labeled video data and a robust graph co-clustering approach, one can uncover visual patterns of un-usual and usual actions. This was an exciting discovery, as it suggested that big-data can solve this hard vision problem without explicitly defining action categories, and without detailed analysis of human motion. Through more experiments, it was clear that such big-data approach has an ‘autistic’ limitation: it memorizes many details, but understands little intricate relationships of human motion and causality among them. It has little ability to make long-range prediction of future actions. My recent work on human action recognition is aimed to resolve this ‘autistic’ limitation. “Action = motion + intention”. Intention as And-Or graph of Actor-Actions: Storyline model. In “Understanding Videos, Constructing Plots: Learning a Visually Grounded Storyline Model from Annotated Video” (cvpr2009), we studied causality among actions. Our Storyline model can be regarded as a stochastic spatio-temporal grammar, whose language (individual storylines) represents potential plausible explanations of new videos in a domain. The basic insight is that all the variations of an event share a goal directed sequence of actions (akin to how movies of the same genre has similar flow of story subplots). The variations, such as one falls down as he runs towards a car, are due to different effects of each action. Our method requires only short video segments accompanied by a text description of the actors and actions present in the video. The system requires no detail annotations of the actor and actions in the video. We model the storyline grammar as a probabilistic AND-OR graph. The Storyline inference